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Abstract 

We study how an irreducible closed algebraic curve X embedded in 
CP^, which degree is d and genus g, can be recovered using its projections 
from points onto embedded projective planes. The different embeddings 
are unknown. The only input is the defining equation of each projected 
curve. We show how both the embeddings and the curve in CP^ can be 
recovered modulo some actions of the group of projective transformations 
of CP^ 

In particular in the case of two projections, we show how in a generic 
situation, a characteristic matrix of the two embeddings can be recovered. 
In the process we address dimensional issues and as a result establish the 
minimal number of irreducible algebraic curves required to compute this 
characteristic matrix up to a finite-fold ambiguity, as a function of their 
degree and genus. Then we use this matrix to recover the class of the 
couple of maps and as a consequence to recover the curve. For a generic 
situation, two projections define a curve with two irreducible components. 
One component has degree d{d— 1) and the other has degree d, being the 
original curve. 

Then we consider another problem. A'^ projections, with known pro- 
jections operators and A'' >> 1, are considered as an input and we want 
to recover the curve. The recovery can be done by linear computations in 
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the dual space and in the Grassmannian of lines in CP'*, that we denote 
by G(l, 3). Those computations are respectively based on the dual variety 
and on the variety of intersecting lines. In both cases a simple lower bound 
for the number of necessary projections is given as a function of degree 
and genus. A closely related question is also considered. Each point of a 
finite closed subset of an irreducible algebraic curve, is projected onto a 
plane from a point. For each point the center of projection is different. 
The projections operators are known. We show when and how the recov- 
ery of the algebraic curve is possible in function of the degree of the curve 
of minimal degree generated by the centers of projection. 

These questions were motivated by applications to static and dynamic 
computational vision. Therefore a second part of this work is devoted to 
applications to this field. The results in this paper solve a long stand- 
ing problem in computer vision that could not have been solved without 
algebraic-geometric methods. 



1 Introduction 

Consider an irreducible closed algebraic curve X e CP"^ (in the sequel we simply 
write P" for CP"). This curve is projected onto several projective planes em- 
bedded in P'^ through several center of projections, say {0}i,i = 1, Each 
projection mapping, denoted by tTj : P^ \ {0^} — > P^ is presented as a 3 x 4 
matrix, defined modulo multiplication by non-zero scalar. Then each point 
P different from is mapped by tt^ to MjP. Each projection operator tt^, 
via its matrix, can be regarded as a point in P^^. Let Yi — TTi{X) be the dif- 
ferent projections of the curve X. In the sequel we always deal with generic 
configurations, even when not mentionned explicitly. 

When we consider the problem of recovering the projections maps from the 
projected curves, we will show that the recovery is possible only modulo some 
action of the group of projective transformations of P^ on the set of projection 
maps. To define this action we refer to a projection map as a point in P^^. 
Assume that we have n projections, consider the following projective variety 



V =P^^ X ... X P^^. Let Prs be the group of projective transformations of P^. 
We define an action of Pra on V as follows: On ■ Pr^ — > Mor(V,V),A i— > 
((Qi,...,Q„) ^ (MiA"^, M„A~^)), where each matrix Mj is built from 
the coordinates of Qj = [Qi,!, ■■■TQi,i2V follows: 



Vi,2 Vi,3 <Vi,4 
Qi,5 Qifi Qi,7 Qi,8 
Qi,9 Qi,10 Qi,ll Qi,12 



The geometric meaning of this action is that if we change the projective basis 
in P^, by the transformation A, we need to change the projection maps in 
accordance for the projected curves to be invariant. 

We first investigate the case of two projections. Given the projected curves 
Yi and Y2 as the only data, our first problem is to compute the characteristic 
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matrix (to be defined below) of two projections maps, tti and 7r2, up to a finite- 
fold ambiguity. It is shown that this is equivalent to find a necessary and 
sufficient conditions on X for the action of 62 to have a finite number of orbits. 
Then we show that for each orbit we can recover the curve X modulo Pr^ . More 
precisely each orbit induces a curve embedded in P'^ containing two irreducible 
components, one of degree d{d — 1) and the other of degree d. The latter is the 
curve we are looking for. 

Then we turn to another problem. The projections maps tt^, z = 1, .., n are 
now assumed to be known, in addition to the projected curves Yi. We want to 
recover the curve X . This can be performed by linear computations using either 
the dual varieties or the variety of lines intersecting X . In both case a simple 
lower bound of the minimal number of projections is simply deduced. 

With the variety of lines intersecting X another problem is also handled. 
Consider a set of N points in P"^ and let X be the curve generated by these 
points. Each of these points is projected on a different plane by a different 
projection operator tt^. Those projection operators are known. We want to 
recover X by linear computations. Let Z be the curve, of minimal degree, 
generated by the centers of projections. We give a formula of the number of 
constraints obtained on X as a function of the degree of Z. 

Finally we show how those questions were motivated by some problems re- 
lated to static and dynamic computational vision. Therefore we conclude by 
showing how our results can be applied in that context. 

Since our computations will occur in P"^, we fix [X, Y, Z,T]'^ , as homogeneous 
coordinates, and T = as the plane at infinity. 



2 Projection operators 



Let TT be a projection operator from P'^ to an embedded projective plane i(P^) 
through a point O. This projection can be presented by a 3 x 4 matrix M. There 
exists a set of simple, but very useful, properties. The kernel of M is exactly 
the center of projection. The transpose of M maps a line in i(P^) to the plane 
it defines with the center of projections, given as point of the dual space P^*. 
This can easily be deduced by a duality argument and a simple computation. 

There exists a matrix M, being a polynomial function of M, which maps a 
point in i(P^) to the Pliicker coordinates of the line it generates with the center 
of projections. If the matrix M is decomposed as follows: 



M = 



then for p = [x, y, z]^, the line Lp = Mp is given by the extensor: Lp = 
xA A 9 + y9 A F + zF A A, where A denotes the meet operator in the Grassman- 



Cayley algebra (see 
lines in i(P^). 



|). By duality, the matrix M = M-^ maps lines in P'^ to 
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Consider now two projection operators tti and it2- Let Oi and O2 be the 
center of projections and h(P^) and Z2(P^) the plane of projections. Let be 
the point of intersection of «j(P^) with the hne Oi02. Let (j(e2) be the pencil 
of lines in i2(P^) through 62. It is easy to define a map from ii(P^) \ {ei} to 
(7(62) as follows. Each point p is sent to the line given by 7r2(7rf "^(p)). This 
map is linear and its matrix is F = M2M1. Following the standard terminology 
used in computational vision, we will call the matrix F the fundamental matrix 
of the pair of projections tti and 1^2 and the points ei and 62 will be respectively 
called the first and the second epipole. The line in the first (second) projection 
plane passing the first (second) epipole atre called the epipolar lines. Clearly 
Fei = ejF = 0. 

Proposition 1 The knowledge of the fundamental matrix F of a couple of pro- 
jections operators allows the recovery of the matrices Mi and M2 of the projec- 
tions modulo the action 62- More precisely the couple (Mi,M2) is equivalent to 
([1,0], [H, 62]), where H = — j|^^F. The matrix [02] is defined as being r(e2) 
where t maps any vector x of to the matrix that represents the cross-product 
by X. We have: 

-X3 X2 

t(x) = [x] = X3 -Xl 

-X2 Xl 

and X = [xi,X2,X3]. 

Proof: We are looking for a matrix A G Pra such that: 

Ml = [I,0]A-i 
M2 = [H,e2]A-i 



Let us write B 



as follows: 



B = 



u 







Let us write Mj = [Mj,mi]. Then it follows immediately from the definition 

of F that F = [e2]M2Mi . Using the following algebraic identity for any two 
vectors x and y in C'^: (x^y)I = xy^ 
sufficient to take: fl = Mi, u = mi, v 



[x][y], it is easy to prove that it is 
T-^M2e2 and A = ^(el^m2 - 



"Hmi). 



This shows that in order to characterize a couple of projections modulo the 
action 62, we only need to compute the fundamental matrix. In the sequel we 
show how to recover F from two projections of an algebraic curve. 
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3 Two projections with unknown projection op- 
erators 



In this section we deal with the first problem. A smooth and irreducible curve 
X embedded in P'^ is projected onto two generic planes through two generic 
points. The projection operators tti and n2 are unknown. First we want to 
recover the fundamental matrix of the couple (tti, 7r2) from the projected curve 
Fi = TTiiX) and Y2 - MX). 



3.1 Single projection 

We shall mention of set of well known facts about generic projection. Let X be 
a smooth irreducible algebraic embedded in P"^ and Y its projection on a generic 
plane through a center of projections, O. 

1. The curve Y will always contain singularities. Furthermore for a generic 
position of the center of projection, the only singularities of Y will be 
nodes. 

2. The class of a planar curve is defined to be the degree of its dual curve. 
Let m be the class of Y. Then m is constant for a generic position of the 
center of projections. 

3. If d and g are the degree and the genus of X, they are respectively, for 
a generic position of O, the degree and the genus of Y, and the Pliicker 
formula yields: 

m = 1) - 2(t|nodes), 
9=- '-^ ^-(ttnodes), 

where jinodes denotes the number of nodes of Y . Hence the genus, the 
degree and the class are related by 

m^2d + 2g - 2. 



3.2 Fundamental matrix construction 

We are ready now to investigate the recovery of the fundamental matrix of a 
couple of projections (tti , 1:2) when the only knowledge is made of the projections 
of a smooth irreducible curve. 

As before, let X be a smooth irreducible curve embedded in P"^, which cannot 
be embedded in a plane. The degree of X is d > 3. Let M^, i = 1,2, be the 
projection matrices. Let F, ei and 62 be defined as before. We will need to 

consider the two following mappings: p i— > ei V p and p 1-^ Fp, where V is the 
join operator Q, which is equivalent to the cross-product in that case. Both 
maps are defined on the first projection plane. 
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Let Yi and Y2 be the projected curves. Assume that they are defined by 
the polynomials /i and f2- Let Y* and Y2 denote the dual curves, whose 
polynomials are respectively (pi and 02. 

Theorem 1 For a generic position of the centers of projections with respect to 
the curve X , there exists a non-zero scalar X, such that for all points p in the 
projection plane, the following equality holds: 

Mi(p)) = ^Miip)) (1) 

For reasons that will be clear later, we shall call this equation, the generalized 
Kruppa equation. 

Proof: Let be the set of epipolar lines tangent to curve in image i. We start 
with the following lemma: 

Lemma 1 The two sets ei and £2 are projectively equivalent. Moreover for each 
corresponding pair of epipolar lines (1, 1') G ei x e2, the multiplicities ofl and V 
as points of Yj* and Y2 are the same. 

Proof: Consider the following three pencils: 

• (t(L) « P^, the pencil of epipolar planes, that is planes containing the 
baseline joining the two centers of projection, 

• cr(ei) « P^, the pencil of epipolar lines in the first projection plane, 

• CT(e2) ~ P^, the pencil of epipolar lines in the second projection plane. 

Thus we have C a{ei). Moreover if E is the set of planes in ct(L) tangent to 
the curve in space, then there exists a one-to-one mapping between E and each 
ei which leaves the multiplicities unchanged. This completes the proof. ■ 
This lemma implies that both sides of equation |l| define the same algebraic 
set, that is the the union of epipolar lines tange nt to Yi. Since (t>i and (f)2, in the 
generic case, have the same degree (as stated in ^t] ), each side can be factorized 
as follows: 

0i(7(x, y, z)) = Y]_{aux + a2iy + a^izY' 

i 

4>2i^{x, V, z)) = J]^ \{aux + a2iV + azizf' , 

i 

where '^i ~ — "m-- By the previous lemme we also have: ai = bi for all 

i. m 
By eliminating the scalar A from the generalized Kruppa equation (|l|) we 
obtain a set of bi- homogeneous equations in F and ei. Hence they define a 
variety in P^ x P^. We turn our attention to the dimensional analysis of this 
variety. Our concern is to exhibit the conditions for which this variety is discrete. 
From a practical point of view, this is a step toward the recovery of the original 
curve. 
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3.3 Dimension analysis 

Let {Ei{F, ei)}i be the set of bi-homogeneous equations on F and ei, extracted 
from the generahzed Kruppa equation (|^). Our first concern is to determine 
whether aU solutions of equation (|l|) are admissible, that is whether they satisfy 
the usual constraint Fei = 0. Indeed we prove the following statement: 

Proposition 2 As long as there are at least 2 distinct lines through ei tangent 
to Yi, equation ^) implies that rankF = 2 and Fei = 0. 

Proof: The variety defined by (j>i{j{p)) is then a union of at least 2 distinct 
lines through ei. If equation (Q) holds, (t>2{£.{p)) must define the same variety. 

There are 2 cases to exclude: If rankF = 3, then the curve defined by 
(f>2{£,{p)) is projectively equivalent to the curve defined by 02, which is Y^. In 
particular, it is irreducible. 

If rankF < 2 or rankF = 2 and Fei 7^ 0, then there is some a, not a multiple 
of ei, such that Fa = 0. Then the variety defined by 02(C(p)) is a union of lines 
through a. In neither case can this variety contain two distinct lines through 
ei, so we must have rankF = 2 and Fei =0. ■ 

As a result, in a generic situation every solution of {Ei{F, ei)}i is admissible. 
Let V be the subvariety of x x P^ defined by the equations {Ei{F,ei)}i 
together with Fei = and e2"^F = 0-^, where e2 is the second epipole. We next 
compute a lower bound on the dimension of V, after which we will be ready for 
the calculation itself. 

Proposition 3 IfVis non-empty, the dimension of V is at least 7 ~ m. 

Proof: Choose any line 1 in P^ and restrict ei to the afFine piece P^\l. Let(x,y) 
be homogeneous coordinates on L If Fei — 0, the two sides of equation (|l|) are 
both unchanged by replacing p by p + aei . So equation (|^) will hold for all p 
if it holds for all p 6 1. Therefore equation ([|) is equivalent to the equality of 2 
homogeneous polynomials of degree m in a; and y, which in turn is equivalent 
to the equality of (to+ I) coefficients. After eliminating A, we have m algebraic 
conditions on (ei,F,e2) in addition to Fei = 0, e2"^F — 0^ . 

The space of all epipolar geometries, that is, solutions to Fei = 0, e2"^F = 
0"^, is irreducible of dimension 7. Therefore, V is at least (7 — TO)-dimensional. 
■ 

For the calculation of the dimension of V we introduce some additional 
notations. Given a triplet (ei,F, e2) G P^ x P* x P^, let {qic((ei)} (respectively 
{q2a(e2)}) be the tangency points of the epipolar lines through ei (respectively 
e2) to the first (respectively second) projected curve. Let QQ(ei,e2) be the 3D 
points projected onto {qia(ei)} and {q2a(e2)}. Let L be the baseline joining 
the two centers of projections. We next provide a sufficient condition for V to 
be discrete. 

Proposition 4 For a generic position of the centers of projection, the variety 
V will be discrete if, for any point (ei, F, e2) G V , the union o/L and the points 
QQ(ei,e2) is not contained in any quadric surface. 
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Proof: For generic projections, there will be m distinct points {qia(ei)} and 
{q2a(e2)}, and we can regard Qiq, q2a locally as smooth functions of ei, 62. 

We let W be the affine variety in x x C"^ defined by the same equations 
as V. Let 8 = (ei, F, 62) be a point of W corresponding to a non-isolated point 
of V. Then there is a tangent vector d = (v, $, v') to at 8 with $ not a 
multiple of F. 

If X is a function on W, Ve,ij(x) "^iH denote the derivative of x in the 
direction defined by i9 at 8. For 

XQ(ei,F,e2) = q2a(e2)'^FqiQ(ei), 

the generalized Kruppa equation implies that Xa vanishes identically on VF, so 
its derivative must also vanish. This yields 

Ve,^(XQ) = (Ve,^(q2Q))'^FqiQ 

+ qL^qia + qLF(Ve,.j(qia)) (2) 
0. 

We shall prove that Ve.ij(qia) is in the linear span of qic and ei. (This 
means that when the epipole moves slightly, qia moves along the epipolar line.) 
Consider K{t) = /(qiQ(ei + iv)), where / is the polynomial defining the image 
curve Yi. Since qiQ(ei + tv) e Fi, k = 0, so the derivative k'(0) = 0. On the 
other hand, k'(0) = Ve,i;(/(qia)) = gradq^^ (/)^Ve,i>(qia)- 

Thus we have gradq^^ (/)^Ve,i>(qiQ) = 0. But also gradq^^ (/)^qia = 
and gradq^^(/)-^ei = 0. Since gradq^^(/) ^ O (qiQ, is not a singular point of 
the curve), this shows that Ve,i?(qiQ), qia, and ei are linearly dependent, qi^ 
and ei are linearly independent, so Ve,i9(qia) must be in their linear span. 

We have that qLFei = q^a^^ia = Oj so qJaF^e.f (qia) = 0: the third 
term of equation vanishes. In a similar way, the first term of equation 
vanishes, leaving 

qL^qia = 0. (3) 

The derivative of x(ei, F, 62) — Fei must also vanish, which yields 

e2^$ei = 0. (4) 

From equality (||), we deduce that for every Q^, we have 

QjM^$MiQ„ = 0. 

From equality (^, we deduce that every point P lying on the baseline must 
satisfy 

P^M^$MiP = 0. 

The fact that <& is not a multiple of F implies that M|"$Mi ^ 0, so together 
these two last equations mean that the union LU{Qq} lies on a quadric surface. 
Thus if there is no such quadric surface, every point in V must be isolated. ■ 
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Observe that this result is consistent with the previous proposition, since 
there always exist a quadric surface containing a given line and six given points. 
However in general there is no quadric containing a given line and seven given 
points. Therefore we can conclude with the following theorem. 

Theorem 2 For a generic position of the centers of projection, the generalized 
Kruppa equation defines the epipolar geometry up to a finite-fold ambiguity if 
and only if m > 7. 

Since different curves in generic position give rise to independent equations, 
this result means that the sum of the classes of the projected curves must be at 
least 7 for to be a finite set. 

3.4 Recovering the curve 

Let the projection matrices be Mi and M2. Hence the two cones defined by the 
projected curves and the centers of projections are given by: Ai(P) = /i(MiP) 
and A2(P) = /2(M2P). The reconstruction is defined as the curve whose 
equations are Ai — and A2 — 0. This curve has two irreducible components 
as the following theorem states. 

Theorem 3 For a generic position of the centers of projection, namely when 
no epipolar plane is tangent twice to the curve X , the curve defined by {Ai = 
0, A2 = 0} has two irreducible components. One has degree d and is the actual 
solution of the reconstruction. The other one has degree d{d — 1). 

Proof: For a line L C P'^, we write cr(L) for the pencil of planes containing L. 
For a point p G P^, we write cr(p) for the pencil of lines through p. There is a 
natural isomorphism between iT(ej), the epipolar lines in image i, and o'(L), the 
planes containing both centers of projections. Consider the following covers of 
pi; 

1. X o'(L) = P^, taking a point x £ X to the epipolar plane that it 
defines with the centers of projection. 

2. Yi o'(ei) ^ cr(L) — P^, taking a point y G Yi to its epipolar line in 
the first projection plane. 

3. Y2 cr(e2) = cr(L) = P^, taking a point y G Y2 to its epipolar line in 
the second projection plane. 

If Pi is the projection X Yi, then 77 = ijipi. Let B the union set of branch 
points of r]i and 772- It is clear that the branch points of rj are included in 
B. Let S = F^\B, pick t e S, and write Xs = V^^{S), Xt = V^^{t). Let 
HXs be the monodromy: TTi{S,t) — > Perm(Xt), where Perm(Z) is the group 
of permutation of a finite set Z, see It is well known that the path- 

connected components of X are in one-to-one correspondence with the orbits 
of the action of im(fixs) Xf. Since X is assumed to be irreducible, it has 
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only one component and iin{fj,Xs) acts transitively on Xt- Then if im{nxs) is 
generated by transpositions, this will imply that im{^Xs ) — Perm(X(). In order 
to show that im.{fxxs) actually generated by transpositions, consider a loop 
in based at t, say k- If k does not go round any branch point, then k is 
homotopic to the constant path in S and then fJ^Xsiih]) = 1- Now in B, there 
are three types of branch points: 

1. branch points that come from nodes of Yi: these are not branch points of 
V, 

2. branch points that come from nodes of Y2'. these are not branch points of 

3. branch points that come from epipolar lines tangent either to Yi or to ¥2'. 
these are genuine branch points of ?/. 

If the loop It goes round a point of the first two types, then it is still true 
that fj,Xs — 1- Now suppose that It goes round a genuine branch point of 77, 
say b (and goes round no other points in B). By genericity, 6 is a simple two- fold 
branch point, hence fJ-Xsiih]) is a transposition. This shows that im{fixs) is 
actually generated by transpositions and so im(/ixs) = Perm(Xt). 

Now consider X, the curve defined by {Ai = 0, A2 = 0}. By Bezout's 
Theorem X has degree cP. Let x E X. It is projected onto a point yi in Y.^, such 
that 771(2/1) — rj2{y2)- Hence X = Yi Xpi restricting to the inverse image of 
the set S, we have Xs = Xs XsXs- We can therefore identify Xt with Xt x Xt. 
The monodromy can then be given by //^^ (x, — {^Xsi^)^ f^Xsiv))- Since 
im(/iXs) ~ Perm(Xt), the action of im(^j^^) on Xt x Xt has two orbits, namely 
{{x,x)} = Xt and {{x,y)\x ^ y}. Hence X has two irreducible components. 
One has degree d and is X, the other has degree d? ~ d — d{d — 1). ■ 

This result provides a way to find the right solution for the recovery in a 
generic configuration, except in the case of conies, where the two components 
of the reconstruction are both admissible. 

4 The N >> 1 projections problem with known 
projection operators 

Now we turn our attention to the second problem. iV >> 1 projections maps 
given by A'' matrices {Mi}i=i^..._jv are known. Therefore projec- 
tions of an irreducible smooth algebraic curve are also provided. The problem 
is to recover the original curve by linear computations as much as possible. 

4.1 Curve presentation in the dual space 

Let X* be the dual variety of X. Since X is supposed not to be a line, the 
dual variety X* must be a hypersurface of the dual space [|l^. Hence let T 
be a minimal degree polynomial that represents X*. Our first concern is to 
determine the degree of T. 
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Proposition 5 The degree of T is m, that is, the common degree of the dual 
projected curves. 

Proof: Since X* is a hypersurface of P'^*, its degree is the number of points 
where a generic hne in P^* meets X* . By duaUty it is the number of planes 
in a generic pencil that are tangent to X. Hence it is the degree of the dual 
projected curve. Another way to express the same fact is the observation that 
the dual projected curve is the intersection of X* with a generic plane in P^*. 
Note that this provides a new proof that the degree of the dual projected curve 
is constant for a generic position of the center of projection. ■ 
For the recovery of X* from multiple projection, we will need to consider 
the mapping from a line 1 of the projection plane to the plane that it defines 
with the center of projection. Let ^ : 1 i— > M^l denote this mapping. There 
exists a link involving T, /x and (/>, the polynomial of the dual projected curve: 
T(/i(l)) = whenever (/>(!) = 0. Since these two polynomials have the same 
degree (because ^ is linear) and ((> is irreducible, there exists a scalar A such 
that 

t(m(1)) = \4>{\) 

for all lines 1 S P^*. Eliminating A, we get ('"^^) — 1 linear equations on T. 
Since the number of coefficients in T is (™^^), we can state the following result: 

Proposition 6 The recovery in the dual space can be done linearly using at 
least k > "sfvf";^)" projections. 

4.2 Curve presentation in the Grasmmannian of lines of 

p3 

Let G(l, 3) be the Grassmanian of lines of P'^. Consider the set of lines in P'^ 
intersecting the curve X of degree d. This defines a subvariety of 3) which 
is the intersection of 3) with a hypersurface of degree d in P^, given by a 
homogeneous polynomial F, defined modulo the dth graded piece 3))^ of 

the ideal of G(l,3) and modulo scalars. However picking one representative of 
this equivalence class is sufficient to recover entirely without any ambiguity the 
curve X. In our context, we shall call any representative of this class the Chow 
polynomial of the curve. We need to compute the class of F in the homogeneous 
coordinate ring of G(l, 3), or more precisely in its dth graded piece, S'(G(1, 3))d, 
which dimension is Nd = (''+^) - {"^'^t^). 

Let / be the polynomial defining the projected curve, Y. Consider the 
mapping that associates to a point in the projection plane the line it generates 
with the center of projection: : p Mp. The polynomial T{h'{p)) vanishes 
whenever /(p) does. Since they have same degree and / is irreducible, there 
exists a scalar A such that for every point p G P^, we have: 

r(i^(p)) = A/(p). 

This yields (''^^) — 1 linear equations on F. 

Hence a similar statement to that in Proposition H can be made: 
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Proposition 7 The recovery in G(l,3) can be done linearly using at least k > 
1 rfifSd^fSd+i projections. 

4.3 Family of projections operators and finite closed sub- 
set of points 

Consider now a finite collection of points Pi in P^. Each point is projected by a 
different projection map. The N maps are known and so the projected points. 

Let X be the smooth irreducible curve generated by the points Pi and Y 
be the smooth irreducible curve, of minimal degree, generated by the center of 
projections. 

Each projected point pi yiels one linear equation on the variety of intersect- 
ing lines of X, namely T{TT^^{pi)) ~ 0, where F is the Chow polynomial of X 
as before. 

Let d and d' be respectively the degree of X and Y. We compute the number 
of constraints obtained on F from the projected points as a function of d and 
d'. In other words, we want to compute the maximal number of constraints 
that one can extract on a smooth irreducible curve X embedded in P"^ from a 
finite number of lines in the join of X with a known curve Y. 

Proposition 8 The maximal number of constraints is Nd — (/i*'(Op5 {d — d')) — 
h'^{Op5{d — d' — 2)) + 1) Q where Nd — (iiTO(S'(G(l, 3))d) is the dimension of 
the d-th graded piece of homogeneous coordinate ring o/G(l,3). 

Proof: Each projected point generates a line with the center of projection. Let 
Li,....,L„ be these n lines joining X and Y. Let Fx and Fy be the Chow 
polynomial of X and Y respectively. We shall denote by Z(Fx) and Z{Ty) the 
sets where they vanish. Let V = Z(Fx) nZ(Fy) nG(l, 3). For n » 1, we have 

{FeiJ"(p5,0p5(d)) : F(i,)=0,^ = l,...,n} = 
{F e H"{F^, Ops{d)) : Fy = 0} = Wr^id). 

So, we want to compute dim{lY f5 (d)), or equivalently, h^{V, Ov{d)) = h^{Op5 (d)) — 
dim{Ivjs{d)). Since F is a complete intersection of degree (d, d',2) in P^, the 
dimension of /y p5 (d) should be equal to 

/i°(Op5 (d - 2)) + /i°(Op5 (d - d')) - /i"(Cp5 (d-d' - 2)) + 1. 

As a consequence 

h°{V, Ov{d)) =Nd- {h°{Or^{d - d')) - h°{Op5{d - d' - 2)) + l) . 



^As usual, h^\Of5{k)) denotes the dimension of the cohomology group Ops (A;)) (see 

§))• 
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5 Applications to static and dynamic computa- 
tional vision 



The results obtained above were motivated by some applications to computa- 
tional vision. We now proceed to show how these results can be applied to this 
field. We start by a quick survey on linear computational vision. More details 
can be found in |5[ ^. Some of the terminology was introduced before in 
section ||. 

5.1 Foundations of linear computational vision 

Projective algebraic geometry provides a natural framework to geometric com- 
puter vision. However one has to keep in mind that the geometric entities to 
be considered are in fact embedded in the physical three-dimensional Euclidean 
space. Euclidean space is provided with three structures defined by three groups 
of transformations: the orthogonal group Euc^ (which defines the Euclidean 
structure and which is included into the affine group), Af (defining the affine 
structure and itself included into the projective group), and Pr^ (defining the 
projective structure). We fix [X, F, Z, T]^, as homogeneous coordinates, and 
T = as the plane at infinity. 



5.2 A single camera system 

Computational vision starts with images captured by cameras. The camera 
components are the following: 

• a plane TZ, called the retinal plane or image plane; 

• a point O, called the optical centre or camera centre, which does not lie 
on the plane TZ. 

The plane TZ is regarded as a two dimension projective space embedded into 
P'^. Hence it is also denoted by j(P^). The camera is a projection machine: 
TT : P3 \ {O} i(p2),P 1-^ OP n i(p2). The projection tt is determined (up to 
a scalar) by a 3 x 4 matrix M (the image of P being AMP). 

The physical properties of a camera imply that M can be decomposed as 
follows: 

f S Uo 

M= af vo [R;t], 
1 

where (/, a, s, uq, vq) are the so-called internal parameters of the camera, whereas 
the rotation R and the translation t are the external parameters. 
It is easy to see that: 

• The camera centre O is given by MO = 0. 

• The matrix M^ maps a line in i(P^) to the only plane containing both 
the line and O. 
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There exists a matrix M G A46x3(K), which is a polynomial function of 
M, that maps a point p G «(P^) to the line Op (optical ray), represented 
by its Pliicker coordinates in P^. If the camera matrix is decomposed as 
follows: 



M 



then for p = [x, y, z]'^ , the optical ray Lp = Mp is given by the extensor: 
Lp — xA A 8 + yQ A F + 2;r A A, where A denotes the meet operator in 
the Grassman-Cayley algebra (see Q). 

• The matrix M = M-^ maps lines in P'' to lines in i{P'^). 

Moreover we will need in the sequel to consider the projection of the abso- 
lute conic onto the image plane. The absolute conic is simply defined by the 
following equations: 

X^ + Y^ + = 
T = 

By definition, the absolute conic is left invariant by Euclidean transforma- 
tions. Therefore its projection onto the image plane, defined by the matrix lo, is 
a function of the internal parameters only. By Cholesky decomposition to — LU, 
where L (respectively U) is lower (respectively upper) triangular matrix. Hence 
it is easy to see that U = M , where M is the 3x3 matrix of the internal 
parameters of M. 



5.3 A system of two cameras 

Given two cameras, {Oj,'ij{P'^))j=i,2 are their components where ii(P^) and 
i2(P^) are two generic projective planes embedded into P"^, and Oi and O2 
are two generic points in P'^ not lying on the above planes. As in \S.2\ let 
TTj : P3 \ {Oj} -> ij(P^),P ^OjPn ij(p2) be the respective projections. The 
camera matrices are M^, i = 1,2. 



5.3.1 Homography between two images of the same plane 



Consider the case where the two cameras are looking at the same plane in space, 
denoted by A. Let 



be the camera matrices, decomposed as above. Let P be a point lying on A. We 
shall denote the projections of P by Pi = [xi,yi, Zi]'^ = M^P, where = means 
equality modulo multiplication by a non-zero scalar. 
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The optical ray generated by pi is given by Lp^ = xiAi A 9i + j/i9i A Fi + 
ZiFiAAi. Hence P = Lp, AA = cciAi AGi A A + yiGi ATi A A + ziFi A Ai A A. 
Hence P2 = M2P is given by the following expression: p2 = HaPi where: 



This yields the expression of the collineation Ha between two images of the 
same plane. 

Definition 1 The previous collineation is called the homography between the 
two images, through the plane A. 

5.3.2 Epipolar geometry 

Definition 2 Let {Oj,ij{F^),M.j)j=i^2 be defined as before. Given a pair {pi,p2) & 
ii(P-^) X i2(F^), we say that it is a pair 0/ corresponding or matching points if 
there exists P € such that pj ~ T^j(P) for j — 1,2. 

Consider a point p S ii(P^). Then p can be the image of any point lying 
on the fiber 7rj^^(p). The matching point in the second image must lie on 
TT2{'!Ti^{p)), which is, for a generic point p, a line on the second image. Since 
TTi and TT2 arc both linear, there exists a matrix F G A^3x3(IR), such that: 
^(p) — 7r2(7rj^^(p)) = Fp for all but one point in the first image. 

Definition 3 The matrix F is called the fundamental matrix, where as the line 
Ip = Fp is called the epipolar line of p. 

Let ei = O1O2 n n(P^) and e2 = O1O2 n Z2(P^). Those two points are 
respectively called the first and the second epipole. It is easy to sec that Fei = 0, 
since TTi^{ei) = O1O2 and 7r2(Oi02) = e2. Observe that by symmetry F^ is 
the fundamental matrix of the reverse couple of images. Hence F'^e2 = 0. Since 
the only point in the first image that is mapped to zero by F is the first epipole, 
F has rank 2. 

Now we want to deduce an expression of F as a function of the camera 
matrices. By the previous analysis, it is clear that F = M2M1. Moreover we 
have the following properties: 

Proposition 9 For any plane A, not passing through the camera centres, the 
following equalities hold: 



where [62] x is the matrix associated with the cross-product as follows: for 
any vector p, e2 x p = [e2]xP- Hence we have: 



Ha 



ri'(Ai A 61 A A) r|'(ei a Ti a A) V^iVi a Ai a A) 

Ai'(AiAeiAA) Aj(eiAriAA) Ai'(riAAiAA) 
el^(AiAeiAA) 9^^(61 A Ti A A) ei'(riAAiAA) 



1. 



F^NxHa 





[62] X = e23 
-622 



-e23 




e2i 



622 
-621 
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In particular, we have: F = [e2]xHoo, where Hqo is the homography 
between the two images through the plane at infinity. 

2. 

H^F + F^Ha = 0. (5) 



Proof: The first equality is clear according to its geometric meaning. Given a 
point p in the first image, Fp is its cpipolar line in the second image. The optical 
ray Lp passing trough p meets the plane A in a point Q, which projection in 
the second image is HaP- Hence the epipolar line must be 62 VHaP. This gives 
the required equality. The second equality is simply deduced from the first one 
by a short calculation. ■ 

Proposition 10 For a generic plane A, the following equality hold: 

HaBi = 62- 

Proof: The image of ei by the homography must be the projection on the second 
image of the point defined as being the intersection of the optical ray generated 
by ei and the plane A. Hence HaBi = M2(Lei A A). But Lgi = O1O2. Thus 
the result must be M2O1 (except when the plane is passing through O2) that 
is the second epipole 62- ■ 



5.3.3 Canonical stratification of the reconstruction 

Three-dimension reconstruction can be achieved from a system of two cameras, 
once the camera matrices are known. However a typical situation is that the 
camera matrices are unknown. Then we face a double problem: recovering the 
camera matrices and the actual object. There exists an inherent ambiguity. 
Consider a pair of camera matrices (Mi, M2). If you change the world coordi- 
nate system by a transformation V e Pr^, the camera matrices are mapped to 
(MiV"^, M2V~^). Therefore we define the following equivalence relation: 

Definition 4 Given a group of transformations G, two pairs of camera matri- 
ces, say (Mi,M2) and (Ni,N2), are said to he equivalent modulo G if there 
exists V e G such that Mi = VNi and M2 = VN2. 

Any reconstruction algorithm will always yields a reconstruction modulo a 

certain group of transformations. More prcsicely there exist three levels of re- 
construction according to the information that can be extracted from the two 
images and from a priori knowledge of the world. 

Projective stratum. 

When the only available information is the fundamental matrix, then the 

reconstruction is done modulo Pr^i. Indeed, from F, the so-called intrinsic ho- 
mography S — — -ip^^F is computed and the camera matrices are equivalent to 
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([I;0],[S;e2]). 



AfRne stratum. 

When, in addition to the epipolar geometry, the homography between the 
two images through the plane at infinity, denoted by Hoc, can be computed, the 
reconstruction can be done modulo the group of affine transformations. Then 
the two camera matrices are equivalent ([I;0], [Hoo;e2]). 

Euclidean stratum. 

The Euclidean stratum is obtained by the data of the projection of the 
absolute conic f2 onto the image planes, which allows the recovery the inter- 
nal parameters of the cameras. Once the internal parameters of the cameras 
are known, the relative motion between the cameras expressed by a rotation 
R and a translation t can be extracted from the fundamental matrix. How- 
ever only the direction of t, not the norm, can be recovered. Then the cam- 
eras matrices are equivalent, modulo the group of similarity transformations, to 
(Ml [I; 0], M2[R; t]), where Mi and M2 are the matrices of internal parameters. 

Note that the projection of the absolute conic on the image can be com- 
puted using some a priori knowledge of the world. Moreover there exist famous 
equations linking cui and UJ2, the two matrices defining the projection of the 
absolute conic onto the images, when the epipolar geometry is given. This is 
the so-called Kruppa equation, defined in the following proposition. 

Proposition 11 The projections of the absolute conic onto two images are re- 
lated as follows. There exists a scalar A such that 

[ei]^c^*[ei]x -AF^c^2*F, 

where [ei]x is the matrix representing the cross-product by ei and lo* is the 
adjoint matrix of uJi. 

Let be the tangents to 7ri(f7) through e;. Kruppa equation simply states 
that ei and e2 are projectively isomorphic. 



5.4 Applications of the previous results to computational 
vision 

The previous results (sections ^ and ^ can be applied to computational vision 
in different contexts: 

The recovery of the epipolar geometry from two images of the same smooth 
irreducible cur ve. Theorem ^ generalizes Kruppa equation to algeraic 



curves. Section provides with a necessary and sufficient conditions on 
the degree of the curve for the epipolar geometry to be defined up to finite 
fold ambiguity. Note that the case of conic sections was first introduced 
in 
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2. The 3D reconstruction of a curve from two images is possible in a generic 
situation as shown in theorem ^. The case of conic was also treated in 
p^ , p^ , Note that |^ presents an algorithm for curve reconstruction 
using a blow-up of the projected curve. This nice result, however, does not 
provide any information about the relative position of the curve in P3 with 
respect to other elements of the scene. On the other hand, our approach 
based on two images allows reconstructing the curve in the context of 
the whole scene. Furthermore the problem of curve reconstruction was 
also considered in |^ from the point of view of global optimization and 
bundle ajustement. Our approach, on the contrary, is based on looking at 
algebraic curves for which the represenation is more compact. 

3. The 3D reconstruction of a curve from A'^ >> 1 projection is linear using 



the dual space or the Grassmannian of lines 3), sections 11 and [4.2| . 
The formalism of the dual space in the case of conic or quadric was also 
used in 0, |l§. 

The trajectory recovery of a moving point viewed by a moving camera 
whose matrix is known over time is a linear problem when using the va- 
riety of intersecting lines of the curve generated by the motion of the 
point. Moreover this gives rise to the question of counting the number 
of constraints that can be obtained. This is done in theorem ^. Note 
that our algorithm for trajectory recovery or triangulation is a complete 
generalization of |^]. 



5.5 Experiments and discussion 

Now we are in a position to give some experiments of the different applications 
mentionned above. The algorithms induced by our theoretical analysis involve 
either solving polynomial systems built from noisy data or estimating high- 
dimensional parameters that appear linearly in equations built also from noisy 
data. 

The first of these problems is still under very active research. In our best 
knowledge, one of the most powerful solver is FastGb, introduced by Jean- 
Charles Faugere ^. Our experiments involving solving polynomial systems 
have been conducted using FastGb. However at this stage, we will show only 
synthetic experiments. Future research will be devoted to the case of real data. 
The second question is a typical case of heteroscedastic estimation ||l^ and will 
be discussed below. 

Recovering Epipolar Geometry from a rational cubic and tv^ro con- 
ies 

We proceed to a synthetic experiment, where the epipolar geometry is com- 
puted from a rational cubic and two conies. The curves are randomly chosen, 
as well as the camera. 
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Hence the cubic is defined by the following system: 



226566665956452626ZX - 1914854993236086169ZT - 791130248041963297rZ- 
1198609868087508022^2 + 893468169675527814XT + 285940501848919422T2- 
179632615056970090rT + 2779600382264726561^ = 

555920076452945312Xr + 656494420457765614ZX - 1755155973545148735rZ- 
1749154450800074954^2 + 984240461094724954XT - 61309565864179510yr- 
1802588912007356295ZT+ 291319745776795474r2 = 

1111840152905890624^2 - 2905335341664005486ZX - 793850352563738017^^+ 
1286890161434843658^2 + 1713207647519936006Xr - 248798847306328202rT- 
2942349361064284313ZT+ 39881438695158513472 = 

The first and the second conic are respectively defined by: 

YZ + 30rr + 81^2 - 5ZT - 28r2 



25X + 9y + 40Z + 61T = 
40^2 - 78Xr + mZX + WXT 



88y2 



and 

4X - iir + loz 

-82X2 _ 4gj^y _ 



h57T = 

wzx 







38XT - 7y2 
The camera matrices are given by: 



h^YZ - 94Fr - 68^2 + 14ZT - 35r2 



Ml = 



-87 79 43 -66 
-53 -61 -23 -37 
31 -34 -42 88 



M, = 



-76 
-61 
-66 



-65 
-60 
-32 



25 
9 

78 



28 
29 
39 



Then we form the Extended Kruppa's Equations for each curve. From a 
computational point of view, it is crucial to enforce the constraint that each A 
is different from zero. Mathematically this means that the computation is done 
in the localization with respect to each A. 

As expected, we get a zero-dimension variety which degree is one. Thus there 
is a single solution to the epipolar geometry given by the following fundamental 
matrix: 



511443 



2669337 



23' 



228242 



228242 114121 

Reconstruction of a spatial quartic in P'^ 

Consider the curve X , drawn in figure ^ defined by the following equations: 

F2{x, y, z, t) — xt — {z — 10t)2 

The curve X is smooth and irreducible, and has degree 4 and genus 1. We 
define two camera matrices: 



Ml = 



5 

1 -2 
-10 



M 



2 — 



1 -10 
0-1 
1 -10 
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Figure 1: A spatial quartic 

Then the curve is reconstructed from the two projections. As expected there 
are two irreducible components. One has degree 4 and is the original curve, while 
the second has degree 12. 

Reconstruction using the Grassmannian 

For the next experiment, we consider six images of an electric wire — one 
of the views is shown in figure ^ and the image curve after segmentation and 
thinning is shown in figure ^. Hence for each of the images, we extracted a set 
of points lying on the thread. No fitting is performed in the image space. For 
each image, the camera matrix is calculated using the calibration pattern. Then 
we proceeded to compute the Chow polynomial F of the curve in space. The 
curve X has degree 3. Once F is computed, a reprojection is easily performed, 
as shown in figure ^ 

The computation of the Chow Polynomial involves an estimation problem. 
Moreover as mentioned above, the Chow polynomial is not uniquely defined. In 
order to get a unique solution, we have to add some constraints to the estimation 
problem which do not distort the geometric meaning of the Chow polynomial. 
This is simply done by imposing to the Chow polynomial to vanish over Wd 
additional arbitrary points of which do not lie on G(l,3). The number of 
additional points necessary to get a unique solution is Wd — ~ Nd, where 

d is the degree of the Chow polynomial. 

As we shall the estimation of the Chow polynomial is a typical case of het- 
eroscedastic estimation. Every 2D measurement p is corrupted by additive 
noise, which we consider as an isotropic Gaussian noise JV{0,<7). The variance 
is estimated to be about 2 pixels. 

For each 2D point p, we form the optical ray it generates L — Mp. Then 
the estimation of the Chow polynomial is made using the optical rays L. In 
order to avoid the problem of scale, the Pliicker coordinates of each line are 
normalized such that the last coordinate is equal to one. Hence the lines are 
represented by vectors in a five-dimensional affine space, denoted by . Hence 
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if is a vector containing the coefficient of the Chow polynomial F, 9 is solution 
of the following problem: 



Z(La)"^0 = 0, for all optical rays, 

with [| 9 11= 1 and ZiJLa) is a vector which coordinates are monomials generated 
by the coordinates of La- Following in order to obtain a reliable estimate, 

the solution 9 is computed using a maximum likelihood estimator. This allows 
to take into account the fact that each Z(La) has a different covariance matrix, 
or in other terms that the noise is heteroscedastic. More precisely, each Z(Lia) 
has the following covariance matrix: 



Cl = J0J„M 



cr 
cr 





where M is the camera matrix and J„ and are respectively the Jacobian 
matrices of the normalization of L and of the map sending to Z(La). That 
is for L(i) = [Li, L2, L3, L4, L5, LqY , we have: 

'F 



1 

















1 

















1 

La 

















1 

















1 



and is similarly computed. Then we use the method presented in Q to per- 
form the estimation. It is worth to note that the estimation is reliable because 
the initial guess of the algorithm was well chosen and because the number of 
measurements is very large. It is necessary to use a very large number of mea- 
surements for two reasons. First the dimension of the parameter space is quite 
high and secondly the measurments are concentrated on a part of the space 
(over the Grassmannian (G(l, 3)). 

Synthetic Trajectory Triangulation 

Let P e P'^ be a point moving on a cubic, as follows: 



i3 

+ t + \ 
+ t + 2 



It is viewed by a moving camera. At each time instant a picture is made, 
we get a 2D point p(t) = [x{t), yit)]^ = i^^, ^fH®]^' ^^ere M^(i) = 
[mi{t) , m.2{t) , m.3{t)] is the transpose of the camera matrix at time t. 



21 




Figure 2: One the six views of an electric thread that were used to perform the 
reconstruction. 



Figure 3: An electric thread after segmentation and thinning. 
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Figure 4: Reprojection on a new image. 



Then we build the set of optical rays generated by the sequence. The Chow 
polynomial is then computed and given below: 

r(Li, Le) = -72LIL3 + Ll- 5L1UL5- 
ISLiLsLe + 57L2L3L5 + A8L2LiL5 - 43X1^2^4- 
IOL1L3L5 + 2lLiL^U - SOLiLiLe - IO8L2L3L6+ 
4IL1L2L5 + 69L1L2L6 - 26L1L2L3 - 36L2LI- 
2IL2LI + 3L3LI - 9LIL5 - I2LIL5 + 6LiLl + 
ALlU + 2QLl - ULl + 8L| - if + lOSiiie- 
I2OLIL5 + 27LlLe - 25LlLe + 57X2^1+ 
84L|i4 + 7LiLl - LIL5 + 3IL1LI + 
5LIL3 + LiLl - IILIL2 + 7LiLl 

From the Chow polynomial, one can extract direclty the locations of the 
moving point at each time instant an image was made. This is done by a two 
steps computation. The first step consists in giving a parametric representation 
of the optical ray generated by the 2D measurment. During the second step, the 
pencil of lines passing through a generic point on the optical ray is considered. 
For this generic point to be of the trajectory, the Chow polynomial must vanish 
over the pencil. This yields a polynomial system in one variable, whose root 
gives the location of the 3D moving point. We show in figure || the recovered 
discrete locations of the point in 3D. 

Trajectory Triangulation from real images 

A point is moving over a conic section. Four static non-synchronized cameras 
are looking at it. We show on figure ^ one image of one sequence. 

The camera matrices are computed using the calibration pattern. Every 
2D measurement p{t) is corrupted by additive noise, which we consider as an 
isotropic Gaussian noise 7V(0, a). The variance is estimated to be about 2 pixels. 

As before the estimation is done from the set of optical rays generated by the 
2D moving point. The estimation is also a case of heteroscedastic estimation, 
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Figure 5: The 3D locations of the point 




Figure 7: The trajectory rendered in the cahbration pattern 

which was handled with the method presented in jij . The result is stable where 
starting with a good initial guess. In order to handle more general situation we 
further stabilize it by incorporating some extra constraints that come from our 
a-priori knowledge of the form of the solution. The final result is presented in 
figure ^ 
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